1. Field of the Invention
This invention relates to computer mass storage devices, in particular, relates to Serial Attached SCSI (Small Computer System Interface) mass storage devices.
2. Description of the Related Art
Most computer systems have mass storage devices where software programs, raw data and outputs are stored on a permanent basis. There are many types of mass storage devices, for example, magnetic disk drives such as hard drives, or optical disk drives such as a CD-ROM or DVD. There are also many ways to access these various types of mass storage devices, i.e. the protocols or interfaces connecting between the host computers and the mass storage devices. One of the simplest ways of connecting between a host computer and a mass storage device is to connect the hard drives directly to the motherboard of the host computer. Most personal computers, such as desktop or laptop computers, have mass storage devices connected to the motherboard directly using an IDE (Integrated Drive Electronics) or ATA (AT Attachment) cable, as shown in FIG. 1. The ATA cable, e.g. cable 120 as shown in FIG. 1, has a very limited length and limited number of drives that can be attached, normally two. The width of the cable and its limited length has created packaging problems for the installation of a large number of disk drives, which problem is exacerbated if additional controllers and cables are added. If the need of the computer storage increases, that is the computer needs more storage, then either the disk has to be replaced with a larger hard drive, which has many problems, or an additional disk must be added, with at least the cabling problems discussed above.
The other leading alternative is SCSI (Small Computer System Interface). SCSI also uses a wide cable with limited lengths. However, SCSI can provide more disk drives for any given controller so the number of drives is not as limited. Again, the cable length and width creates problems.
Instead of having this one to one or one-to-few relationship between a hard drive and a computer, the computers and mass storage devices can have a multiple to multiple relationships through some intermediates. For example, a node computer can connect to a storage server through a network, for example Internet or local area network, as shown in FIG. 2. This storage server can have many hard drives or a disk farm including many different hard drives, DVDs or CD-ROMs. The storage server maintains the data integrity on various hard drives by keeping enough redundancy of the hard drives. The storage server can also manage the scalability of the capacity by adding or removing hard drives to increase or decrease the storage capacity of the disk farm. In the view of the node computer through the network, the storage server and the disk farms is just a large mass storage device. This aggregated and networked storage device arrangement is very reliable and flexible but it can be quite complicated and expensive.
In pursuit of better performance at a lower cost, a new type of interface between computer and mass storage device is being developed. One of such interconnections is called Serial Attached SCSI (SAS). The proposed SAS standard is compatible with the Serial ATA physical interconnect. The specifications can be obtained at the website serialata.org. The current specifications include Serial ATA, Revision 1.0a dated Jan. 7, 2003 and Serial ATA II: Extensions to Serial ATA 1.0, Revision 1.0, dated Oct. 16, 2002, both of which are hereby incorporated by reference. The SAS standard includes three types of protocols: Serial SCSI protocol (SSP), Serial ATA Tunneling protocol (STP), and Serial Management Protocol (SMP). This new standard is developed to operate with the SCSI standard, normally used on large computers for direct attached storage.
A host computer is also called an initiator. The hard drive or an individual piece of mass storage is called a target. A target may be connected to an initiator directly as in FIG. 3, similar to the arrangement as in FIG. 1, or may be connected through an expander, as in FIG. 4. Each initiator may connect to one or more expanders and each expander can connect to a number of targets, as shown in FIG. 4. An expander is a type of switch such that each initiator can connect to a target through a pathway through the expander. The connection between initiators to expanders, and expanders to targets are through the interconnection circuits which are called phys. Each phy contains a transceiver which can transmit and receive data frames or commands at the same time. Each initiator, expander or target may have multiple phys, e.g. as shown in FIG. 5A or 5B. In case where an initiator phy is directly attached to a target phy, the pathway and the physical link are identical. As shown in FIG. 5A, the initiator phy 502 is linked to the target phy 504 directly, so the pathway and the physical link are the same. In this case, the initiator has only one phy 502 and the target only has one phy 504. The pathway is the physical route of a connection. In case where there are expander devices between an initiator phy and a target phy, the pathway consists of all the physical links required to route a frame or a command between the initiator phy and the target phy, as shown in FIG. 5B. The physical links may or may not be using the same physical link rate. A connection is a temporary association between an initiator port and a target port. During a connection, frames or commands from the initiator port are forwarded to the target port and similarly all frames from the target port are forwarded to the initiator port. If multiple potential pathways exist between the initiator ports and the target ports, multiple connections may be established by a port between the following: a) one initiator port to one of multiple target ports; b) one target port to one of multiple initiator ports; or c) one initiator port to one target port. Once a connection is established, the pathway used for the connection may not be changed (i.e., all the physical links that make up the pathway remain dedicated to the connection until it is closed.) Since each connection between an initiator port and a target port is a dedicated connection, at each time, a port can only connect to one other port.
As shown in FIG. 5B, initiator 504 may have two ports 507 and 517, each having phy 510 and 520, respectively. Phy 510 is linked to expander phy 562 and port 561, and phy 520 is linked to expander phy 566 and port 565. Target 515 has one port 527 and two phys 530 and 540, it being a wide port. Target 525 has one port 547 and one phy 550. The target phys 530, 540 are linked to expander phys 564, 568 in port 563 and target phy 550 is linked to expander phy 572 in port 571. Initiator phy 510 may have three pathways, one to each of the target phys 530, 540 and 550. For example, the first pathway is from phy 510 to phy 562 on the expander to phy 564 on the expander to target phy 530; the second pathway is from phy 510 to phy 562 on the expander to phy 568 on the expander to target phy 540; the third pathway is from phy 510 to phy 562 on the expander to phy 572 on the expander to target phy 550. But at any given time, phy 510 can have only one pathway open, i.e. connecting to and communicating with one of the three target phys. Similarly, phy 520 may have a pathway to one of phys 530, 540 and 550, but phy 510 and 520 cannot simultaneously be connected to phys 530 and 540 as that would connect two ports 507 and 517 to one port 527.
An initiator can be connected to any one of the targets when needed with or without the intermediate expanders. The connection is only opened when there is a need to communicate from an initiator to a target. For example as shown in FIG. 5B, the initiator may have any one of the three connections to the three targets if the initiator needs to communicate with that one. When the need is satisfied, then the connection is closed. Therefore, there is a protocol to open and close the connections between initiators and targets.
A connection can be opened by sending an OPEN address frame from a source port to a destination port using one source phy and one destination phy. An AIP primitive is generated and sent by the intermediate expanders to indicate that the intermediate expanders are working to open up the connection, usually one segment at a time. The AIP indicates that the expander is waiting its turn to get the resources for opening the requested connection. If the OPEN address frame reaches the destination, the destination may return either OPEN_accept or OPEN_reject primitives. If the OPEN_accept primitive is received by the source port, then the connection is established between the source port and the destination port, as shown in FIG. 6. After that, data transmission can occur.
After the connection has been opened, each port can send frames to the other port. When either the source port or the destination port finishes transmitting frames, it can initiate the closure of the connection by sending a DONE primitive. Either port can send the first DONE. For simplicity of this discussion, it is assumed that the source port initiates the closure of the connection, but it is understood that the destination port could have initiated and the figures would be reversed.
There are several different ways to end a connection between the source port and the destination port. Some of those methods are more preferable than others. The least preferred method of closing a connection is breaking a connection. After transmitting a BREAK, the source port will ignore all incoming frames except for BREAKs. BREAKing a connection should be avoided because it is not an orderly closure between the two connected ports. The status of any outstanding commands is lost and lengthy recovery processes must occur. Using BREAK to close a connection is like using Ctrl+Alt+Del or unplugging the power cord to shut down a computer.
A preferred method of closing a connection is to send and receive DONE primitives between two connecting ports, as shown in FIG. 7. When all data transmission from the source port to the destination port is finished, the source port will send a DONE primitive to the destination port. The two ports will exchange DONE primitives and then CLOSE primitives to orderly close the connection between the source port and destination port. A CLOSE primitive is used to close a connection of any protocol. The CLOSE primitive is the last primitive exchanged between two ports before the connection is ended. If the connection between a source port and a destination port is closed through an orderly closure, the status of each port and all commands are known after the closure. So the next time the source port or the destination port is needed for other connection, the new connection can be started smoothly and immediately. But if the connection between the source port and the destination port is terminated by a BREAK primitive or some time out, then the status of the outstanding higher level commands may be unknown (although the status of the ports may be known). Before the higher level systems are recovered and reinitialized, those ports are not usable for data transmission.
If for any reasons, the two ports fail to exchange DONE primitives, then the CLOSE primitive cannot be used. Thereafter, the connection will be broken by a BREAK primitive or assumed to be broken.
It is desirable to have a method and an apparatus to reduce the chances of abrupt disconnection between a source port and a destination port. It is desirable to expand the availability of the DONE primitive for various situations when the DONE primitive is not currently available.